New paradoxical games based on Brownian ratchets 
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Based on Brownian ratchets, a counter-intuitive phenomenon has recently emerged - namely, 
that two losing games can yield, when combined, a paradoxical tendency to win. A restriction of 
this phenomenon is that the rules depend on the current capital of the player. Here we present new 
games where all the rules depend only on the history of the game and not on the capital. This new 
history-dependent structure significantly increases the parameter space for which the effect operates. 
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In the early 1990's it was shown that a Brownian par- 
ticle in a periodic and asymmetric potential moves to 
the right (say) in a systematic way when the potential is 
switched on and off, either periodically or randomly [EL^] . 
This so-called flashing ratchet is in the class of phe- 
nomena known as Brownian ratchets ||. The flashing 
ratchet can be viewed as the combination of two dynam- 
ics: Brownian motion in an asymmetric potential and 
Brownian motion on a flat potential. In each of these 
two cases, the particle does not exhibit any systematic 
motion. However, when they are alternated the particle 
moves to the right. The effect persists even if we add a 
uniform external force pointing to the left. In that case, 
the two dynamics discussed above yield motion to the 
left but when they are combined, the particle moves to 
the right. 

It has recently been shown, in the seminal papers [Q-Q , 
that a discrete-time version of the flashing ratchet can 
be interpreted as simple gambling games. Here we have 
two losing games which become winning when combined. 
These games are the simplest situation of a paradoxical 
mechanism which, we believe, can be present in many 
situations of interest. The apparent paradox points out 
that if one combines two dynamics in which a given vari- 
able decreases the same variable can increase in the re- 
sulting dynamics. Examples of related phenomena in- 
clude enzyme transport analyzed by a four-state rate 
model ||, finance models where capital grows by in- 
vesting in an asset with negative typical growth rate 
stability produced by combining unstable systems [10|, 
counter- intuitive drift in the physics of granular flow [11|, 
the combination of declining branching processes produc- 
ing an increase fl^| and counter-intuitive drift in switched 
diffusion processes in random media [^i. 

The games originally described in [§|"t7[ are expressed 
in terms of tossing biased coins. The games rely on a 
state-dependent rule based on the player's capital and 
two losing games can surprisingly combine to win. This 
effect was shown to be essentially a discrete-time Brow- 
nian ratchet Q|. This is of interest to information the- 
orists who have long studied the problem of producing 
a fair game from biased coins Q and winning games 
from fair games fa, inspired by the work of von Neu- 



mann [16) - the games we are discussing go a step further, 
demonstrating a winning expectation produced from los- 
ing games and have recently been analyzed from the point 
of view of information theory |l7). Seigman fl^jnj has 
reinterpreted the capital of the games in terms of elec- 
tron occupancies in energy levels, recasting the problem 
in terms of rate equations. Similarly, Van den Broeck et 
al have likened the analysis of the transition probabili- 
ties of the games to Onsager's treatment of reaction rates 
in circular chemical reactions J2(J . It has been suggested 
m D that an area of interest to quantum information 
theory would be to recast the games in term of quantum 
probability amplitudes along the lines of pl|-p3||. Quan- 
tum ratchets have now been experimentally realised p4| ] 
and thus quantum game theory based on ratchets is of 
interest. 

However, one of the limitations the game paradox and 
its applicability to further situations is that it relies on a 
modulo rule based on the capital of the player. The mod- 
ulo arithmetic rule is quite natural for an interpretation 
of the paradox in terms of energy levels, say, however 
for processes in biology and biophysics it is unnatural. 
Applicability of the paradox to population genetics, evo- 
lution and economics has been suggested |25|] and thus a 
desirable version of the paradox would be to have rules 
independent of capital. 

In this letter we present a new interpretation of the 
paradox in terms of good and bad biased coins which 
are played more or less often when the two games are 
combined. This interpretation allows us to introduce an 
important modification to the original games, namely, 
games which do not depend on the capital but only on 
the recent history of wins and loses. 

The two original games are as follows. The player has 
some capital X(t), t = 0, 1,2, . . . In game A the capital 
is increased by one with probability p and decreased by 
one with probability 1 — p. In game B, the rules are: 





Prob. of win 


Prob. 


of loss 


x(t)/3 e z 


Pi 


1 - 


-Pi 


X(t)/3 i z 


P2 


1 - 


-Pi 



Here "win" means increasing the capital by one and 
"loss" decreasing it by one. For the choice, p = 1/2 — e, 
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Pi = 1/10 — e, and p 2 — 3/4 — e, with e > 0, the two 
games have a tendency to lose. More precisely (X(t)) is 
a decreasing function of the number of runs t. However, if 
in each run we randomly choose the game we play, then, 
for e small enough, (X(t)) is an increasing function of t. 

An explanation of this paradox is as follows. First, let 
us imagine the above rules as implemented by three bi- 
ased coins, A, Bi and B 2 , with probability for tails p, 
pi and p2, respectively. We see that A and B\ are "bad 
coins," whereas B 2 is a "good coin" for the player. When 
game B is played alone, at first sight one would say that 
B\ is used one third of the time. However, this is not the 
case. When the capital is multiple of three, X(t) = 3n, 
there is a high probability of losing, i.e., X(t + 1) = 'in — 1 
is the most likely value for the capital at t + 1. If this 
is the case, we have to use coin B 2 in the t + 1 run 
and the most likely outcome is now a win. Therefore, 
the most likely capital at t + 2 is again X(t + 2) = 3n. 
We see that the probability of X(t) being multiple of 
three is bigger than 1/3, due to the very rules of game B. 
The precise value of the equilibrium probability can be 
calculated by defining the Markov process Y(t) = X(t) 
mod 3, which only takes on three values, Y(t) = 0, 1, 2. 
The stationary distribution for Y(t), when e = is given 
by: tt — j^; m — j^; n 2 — The fairness of the 
game is indicated by ttqPi + (tti + K2)P2 = 1/2. 

When coin A comes to play, the stationary distribution 
changes. For instance, if games A and B are switched at 
random, one has: n' — = l^|; n' 2 = |g|. The 

game is no longer fair because ir' Q = 245/709 = 0.346 is 
closer to 1/3 than ttq — 0.385, for the "bad coin" and now 
the "good coin," B 2 , is played more often than before. 
The effect persists even if coin A is bad, leading to the 
paradox. 

This interpretation helps us to find a new version of 
the paradox with capital independent games. Game A is 
the same as before and we introduce game B' which is 
played with four coins: B[, B' 2 , B' 3 , and B' 4 . Which coin 
is used now depends on the history of the game: 



Before last 


Last 


Coin 


Prob. of win 


Prob. of loss 


t-2 


t - 1 




at t 


at t 


loss 


loss 


B[ 


Pi 




loss 


win 


B' 2 


P2 


1—P2 


win 


loss 


B' 3 


P3 


1 ~P3 


win 


win 


B\ 


Pi 


I- Pi 



This is in fact the most general game depending on the 
outcome of the two last runs. The paradox could even be 
reproduced with this type of game if the "bad" coins in 
game B' are played more often than what is expected in 
a completely random game, ie. one quarter of the time. 

Notice that the capital X(t) in game B' is not a Marko- 
vian process. However, one can define the vector 



Y(t) 



X(t) - X(t-1) 
X(t-l)-X(t-2) 



(1) 



which can take four values (±1,±1), and does form a 
Markov chain. The transition probabilities are easily 
obtained from the rules of game B'. Let ni(t), iT2(t) : 
TTz{t) and TTi{t) be the probabilities that Y(t) is (— 1, — 1), 
(1,-1), (—1,1), and (1,1), respectively. The proba- 
bility distribution 7?(£) verifies the evolution equation: 
7?(t + 1) = A7?(£), where the matrix A is given by the 
transition probabilities and reads: 



A = 



1-Pi l-p 3 

Pi p 3 

l-p 2 l-p 4 

P2 Pi 



(2) 



The stationary distribution 7r st of this Markov chain is 
by definition invariant under the action of the matrix A, 
i.e., 7? st A = 7r st . This distribution reads: 



(l-p 3 )(l-p 4 ) 

1| (l-Pi)Pi 
N \ (l-p*)pi 

PlP2 



(3) 



where N is a normalization constant. 

In the stationary regime, the probability to win in a 
generic run is: 



^ 7Tst,i 



Pl(P2 +1-P4) 



(1 - Pi){2pi + 1 - p 3 ) + pip 2 



(4) 



which can be rewritten as p W i n = 1/(2 + c/s), with 
s = P\(j>2 + 1 ~ Pi) > for any choice of the rules, 
and c = (1 -Pi)(l -pa) - piP2- 

Therefore, the tendency of game B' obeys the follow- 
ing rule: if c < 0, B' is winning; if c = 0, B' is fair; 
and if c > 0, B' is losing. Again, here losing, winning 
and fair means that (X(t)) is, respectively, a decreasing, 
increasing or constant function of t. 

Since when game B' is combined with game A the vec- 
tor Y(t) as defined in Eq. (0) is still a Markov chain, the 
same procedure applies. The probabilities of winning are 
now replaced by p\ = (pi +p)/2. Summarizing, to re- 
produce the paradox with capital independent games we 
have to find a set of five numbers, p and pi (i — 1, 2, 3, 4), 
such that 



(2 



1 -p > p 

(1 -p 4 )(l -P3) > PlP2 
Pi -p)(2 -p 3 -p) < (pi +p)(p 2 



■P), 



(5) 



where the third equation is just the second with p\ and 
the inequality reversed (to make the combined game win- 
ning instead of losing) . 

One of the coins in game B' must be "bad" and used 
more often than one quarter of the time. It cannot be 
either B[ or B' 4 because the probability of using these 
coins depends on whether the game is losing or win- 
ning (if B[ is played more often than B' 4 , it is obvi- 
ous that the game is losing). The bad coins should be 
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B' 2 and B' 3 . Let us set p = 1/2 - e, pi = 9/10 - e, 
P2 = P3 = 1/4 — e, and p4 = 7/10 — e. With these num- 
bers, one can see that the two first inequalities in Eq. (|5|) 
are always satisfied if e > 0, whereas the third is sat- 
isfied if e < 1/168 = 0.00595 - ie. the paradox occurs 
when < e < 1/168, for our chosen parameter set in this 
example. 
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FIG. 1. Evolution of capital with play. The lower two 
curves show that games A and B' lose when individually 
played. [2,2] indicates game A played 2 times followed by 
game B' played 2 times and so on. The top curve indicates 
random switching between games A and B'. Capital surpris- 
ingly increases in the random or periodic cases. Simulations 
are carried out with e = 0.003, with averaging over 500 000 
ensembles, p = 1/2 — e, p% = 9/10 — e, p2 — pz — 1/4 — e, 
and p4 = 7/10 — e. There are four possible initial conditions 
— these affect the offsets but not the slopes — all the above 
curves are the average of the four cases 



The simulation in Fig. [j] shows that as games A and 
B' evolve individually the capital declines, as expected 
(ie. they are losing games) . On the same graph we see the 
remarkable result that when A and B' are alternated ei- 
ther randomly or periodically, the capital now increases. 
This reproduces the paradoxical behavior first observed 
in the original games 0], but now without state depen- 
dence on capital. The slopes of the curves corresponding 
to game B' and to the random combination can easily 
be calculated as: (X(t + 1)) — (X(t)) = 2p win — 1, with 
Pwin given by Eq. The old and new games have a 
fundamental difference in that the old ones can be inter- 
preted in terms of a random walk in a periodic environ- 
ment (RWPE) jl9| or a Brownian particle in a periodic 
potential, whereas the rules of the present games are ho- 
mogeneous. We could say that the periodic structure of 
the original games has been transferred to the memory of 
the rules in the new games. Therefore, the paradox needs 
at least one of these two ingredients: inhomogeneity or 
non-markovianity |26|. 




FIG. 2. Value of capital after 500 games. Games A and B' 
are periodically mixed. Game A is played a times, followed 
by B' played b times and so on. Games are played with e=0 
and 500 000 ensemble averages have been taken, p = 1/2, 
pi = 9/10, p 2 =P3 = 1/4, and p 4 = 7/10. 

Consider now a periodic combination of games A and 
B'. Fig. H shows the capital after 500 games - where 
game A is played a times and game B' is played b times. 
We can observe that the resulting capital is greater when 
the games are switched more frequently. This behavior 
agrees with with that of the original games [Q . Note that 
in Fig. H changing the value of e only affects the vertical 
capital displacement, thus setting e = pushes the graph 
into the positive region. 




FIG. 3. Parameter space for game B' when p=l/2. We see 
there are four volumes labelled Qi, Q2, Qz and Q4, bounded 
by the inequalities in Eq. (JHJ). The paradox of two losing 
games that win if randomly combined, occurs if the param- 
eters lie within the volumes marked Qi and Q3. In regions 
Q2 and Qa the reverse effect occurs where games A and B 
are individually winning, but the randomized combination is 
losing. 

For the randomized games, we can now observe the 
volume of parameter space for which the paradox takes 
effect, by plotting the surfaces that represent the bound- 
aries of the inequalities in Eq. (^|). This is shown in Fig. |j, 
where for convenience we have set P2 — P3 to produce the 
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graph in three variables. The volumes enclosed by the 
surfaces marked Qi,...,Qi are the regions of parameter 
space for which the paradox takes effect. Regions Q\ and 
Q3 are where two losing games combine to win. On the 
other hand, Qi and Q4 represent the reverse effect where 
two winning games combine to lose. This conjugate re- 
gion can be simply thought of in terms of changing the 
sign of the capital, so that the perspective of the con- 
cepts 'win' and 'lose' reverse. This was observed in the 
original capital-dependent games p7[ , however the conju- 
gate regions were symmetrical. What is now interesting 
is that the new history-dependent games have asymmet- 
rical conjugate regions, as can be seen in Fig. [| 

Another important comparison between the new 
history-dependent games and the original capital- 
dependent games is that the volume of parameter space 
is now bigger. A numerical mesh analysis on Fig. y| re- 
vealed that the new games have a parameter space about 
50 times larger than the original games reported in [p7| . 
For applications such as in biophysics, it is important 
to find such gaming models with large and hence robust 
parameter spaces. Although it appears that the rates of 
winning from the slopes of Fig. l] are about factor of 2 
lower than the original games, this is only the case for the 
particular chosen parameters. The 50 times increase in 
parameter space is favorable for applications in modeling 
evolutionary processes in biology, for example, where a 
weak pay-off can gradually accumulate over a long period 
of time. 

In summary, we have shown that the apparently para- 
doxical effect where two losing games can cooperate to 
win does work with a history based state-dependent rule 
rather than the original restriction of a modulo capital 
based state-dependence. This, together with an increased 
parameter space opens up the phenomenon to a wider 
range of possible application areas. This suggests that fu- 
ture investigation of further types of history-based rules 
and other types of state dependencies may be fruitful. 
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